Repository navigation
fix(vigild): the reaper no longer loses a service's exit status - #1
Merged
Merged
Conversation
As PID 1 (or with --reaper), vigild's reaper drains every exited child with waitpid(-1, WNOHANG) on each SIGCHLD. That collects vigild's own children too, services and exec checks, which tokio waits on: a pidfd tells tokio its child exited, but does not stop another waitpid from collecting it first. When the reaper won, tokio's wait failed with ECHILD and the exit status was lost: a service that exited 0 was reported as `success=false exit_code=1` and handled by its on-failure policy (restarted, or failure-shutdown with exit code 10), and an exec check that passed counted as failed. Seen on a container whose one-shot service runs right as other services start: it exited 0 and was reported failed on two of three starts. The reaper now keeps each status it collects, for 60 s, and reaper::wait_child, which the service actor and the exec probe use in place of Child::wait, takes it from there when the wait fails with ECHILD. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QAe3biqjeQJRMfngB95eyW
Owner
|
thanks for contribution v3.2.2 will be released soon |
daveman1010221
added a commit
to daveman1010221/vigil-rs-nix
that referenced
this pull request
Nov 2, 2026
… goes The fix was merged upstream as git001/vigil-rs#1 and released in v3.2.2, so it no longer needs carrying. Tried on a five-node cluster: every node booted on it, the services came up, and a killed kube-proxy was reported as failed and restarted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A6J8LPYCZWcnSD1dESmJpX
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When vigild runs as PID 1 (or with
--reaper), the reaper drains every exited child withwaitpid(-1, WNOHANG)on eachSIGCHLD. That also collects vigild's own children, services and exec checks, which tokio is waiting on. A pidfd tells tokio its child exited, but it doesn't stop anotherwaitpidfrom collecting the child first. When the reaper wins that race, tokio's wait fails withECHILDand the exit status is lost:success=false exit_code=1and handled by itson-failurepolicy: restarted, orfailure-shutdownwith exit code 10.We hit this on a container whose one-shot service runs while other services are starting: it exited 0, and vigild reported it as failed on two of three starts.
Fix
reaper::wait_childreplacesChild::waitin the service actor and in the exec probe. When the wait fails withECHILD, it takes the status the reaper kept. The reaper records a status right after itswaitpid, so a waiter can seeECHILDjust before the record exists;wait_childkeeps looking for up to 200 ms.Without the reaper,
wait_childis justChild::wait.Tests
The new tests in
reaper.rscollect a child's status withwaitpidbeforewait_childruns, just as the reaper would. They check that each kind of status survives: an exit code, a success and a signal. They also check that the path without the reaper is unchanged, and that old statuses are dropped.Checked locally on Linux with Rust 1.99, running the same steps as CI:
cargo fmt --all --checkandcargo clippy --workspace --all-targets -- -D warningsare clean.cargo test --workspacepasses, except fordo_start_passthrough_log_modeanddo_start_disabled_log_mode. Those two also fail on unchangedmainon this machine (NixOS), because they run/bin/sleep, which NixOS doesn't have. They should pass on CI's Ubuntu runner.🤖 Generated with Claude Code
https://claude.ai/code/session_01QAe3biqjeQJRMfngB95eyW